Unsupervised Feature Selection for High-Dimensional Non-Gaussian Data Clustering with Variational Inference

Clustering has been a subject of extensive research in data mining, pattern recognition and other areas for several decades. The main goal is to assign samples, which are typically non-Gaussian and expressed as points in highdimensional feature spaces, to one of a number of clusters. It is well-known that in such high-dimensional settings, the existence of irrelevant features generally compromises modeling capabilities. In this paper, we propose a variational inference framework for unsupervised non-Gaussian feature selection, in the context of finite generalized Dirichlet (GD) mixture-based clustering. Under the proposed principled variational framework, we simultaneously estimate, in a closed-form, all the involved parameters and determine the complexity (i.e. both model an features selection) of the GD mixture. Extensive simulations using synthetic data along with an analysis of real-world data and human action videos demonstrate that our variational approach achieves better results than comparable techniques. Index Terms Mixture models, unsupervised learning, generalized Dirichlet, model selection, feature selection, Bayesian estimation, variational inference, human action videos. Wentao Fan is with the Department of Electrical and Computer Engineering, Concordia University, QC, Canada H3G 1T7. (email: wenta [email protected]) Nizar Bouguila is with the Concordia Institute for Information Systems Engineering (CIISE), Concordia University, Montreal, QC H3G 1T7, Canada (email: [email protected]). Djemel Ziou is with the Département d’Informatique, Université de Sherbrooke, Sherbrooke, QC J1K 2R1, Canada (email: [email protected]). 2